Energy Minimization with Discontinuities

Many tasks in computer vision can be formulated as energy minimization problems. In this paper, we consider a natural class of energy functions that permits discontinuities. We show that minimizing these energy functions is NP-hard. However, the energy minimization problem can be solved by computing a minimum cost multiway cut on an associated graph. This allows the use of approximation algorithms that produce answers with quality guarantees. We present an efficient approximation algorithm that produces a local minimum even if very large moves are allowed. We apply our method to the tasks of image restoration and stereo. For both tasks we obtain promising results on data with ground truth. 1 Energy minimization in early vision Many early vision problems require estimating some spatially varying quantity (such as intensity, disparity or texture) from noisy measurements. Such quantities tend to be piecewise smooth; they vary smoothly at most points, but change dramatically at object boundaries. Every pixel p ∈ P must be assigned a label in some set L; for motion or stereo, the labels are disparities, while for image restoration they represent intensities. The goal is to find a labeling f : P → L that is both piecewise smooth and consistent with the observed data. ∗The authors are with the Computer Science Department, Cornell University, Ithaca, NY 14853; email: [email protected], [email protected], [email protected]